Virtual reality (VR) systems are used both within and outside of the video game industry. Displays for VR systems, such as those embedded in a VR headset, typically operate at a minimum refresh rate that is suitable for VR applications. For instance, 90 Hertz (Hz) is a common refresh rate for VR displays. In a “live rendering” scenario, a graphics-based application, such as a video game, outputs frames for rendering at a frame rate that matches the refresh rate of the display, meaning that a new frame received from the application (referred to herein as an “actual frame”) is displayed at every screen refresh. Such a live rendering scenario is often referred to as the application “hitting frame rate.”
In practice, an application does not always hit frame rate for various reasons. For example, the application may intermittently drop a frame, and/or the application may temporarily output frames at a slower rate (e.g., 45 frames per second when the ideal frame rate is 90 frames per second). In situations where the application is not hitting frame rate, a technique called “rotation-only re-projection” can be used to replace missing frames with re-projected frames in a way that accounts for the user's head rotation, making it appear to the user as if the application is hitting frame rate. Without re-projection, for example, a deficient frame rate from the application may cause in-game stuttering or hitching. In VR applications, where the user is fully immersed in the virtual environment, the user can become nauseous if the application fails to hit frame rate and there is no re-projection to compensate for the missing frames. Thus, re-projection is a technique that allows for a better user experience when an application is not hitting frame rate. Consider an example where the application is outputting frames at half the ideal frame rate (e.g., 45 frames per second where 90 frames per second is the ideal frame rate). In this example, every other frame can be re-projected using pixel data from the most recently-rendered actual frame to create a re-projected frame that transforms the scene (e.g., through rotation and re-projection calculations) to match the re-projected scene to the user's current head orientation. This makes it look to the user as if the scene is moving in a way that is expected given the user's head rotation, even when re-projected frames are used to compensate for the application failing to hit frame rate.
Although rotation-only re-projection prevents in-game stuttering or hitching, it produces its own unwanted visual artifacts during head rotation, at least in VR systems that use low-persistence displays (e.g., where the display is illuminated for a small fraction of the frame time). For example, rotation-only re-projection can cause a strobing flash to appear at the leading edge of each display panel in the direction of head rotation when the user is rotating his/her head. This strobing flash occurs during head rotation because the pixels at the leading edge of the display panel toggle every frame between an actual frame's valid pixels and solid black pixels in the following re-projected frame. The black pixels occur during head rotation in the re-projected frames because there is no pixel data available from the previously-rendered actual frame to use for the pixels at the leading edge of the display panel for the re-projected frame. Accordingly, when the user rotates his/her head while re-projection is being used, the leading edges of the display panels in the direction of the user's head rotation toggles every frame between black pixels and illuminated pixels, causing an unwanted strobing artifact at the leading edges of the display panels.
Provided herein are technical solutions to improve and enhance these and other systems.